In a complex system, different components work together to function as the complex system. For example, an airplane may have electrical, mechanical and software components that work together for the airplane to land. An engineer may have different options for a given component in the system (e.g., different control systems or different settings for a control system for the landing gear of the airplane). An engineer testing a complex system can construct a test suite that represents different test cases for the system with selections for the different options for each of the components in the system. The test suite can be referred to as a combinatorial test suite in that it tests different combinations of configurable options for a complex system. If there are failures, the test engineer is faced with the task of identifying the option or combination of options that precipitated the failures (e.g., from a table of entries or summary statistics). When there are multiple components in the complex system, it can be difficult to visualize different options for each component and the results of testing those different options.
An engineer may design an experiment with test cases each test case specifying one of different options for each factor of the experiment (e.g., to test a complex system). A screening design, for instance, is useful for determining which active factors in the experiment affect the outcome. Data (e.g., input and response system data) can be used to generate a model (e.g., a machine learning algorithm model). Validations techniques can be used to validate data for generating the model (e.g., a K-fold cross-validation).